Optimal Nonergodic Sublinear Convergence Rate of Proximal Point Algorithm for Maximal Monotone Inclusion Problems

We establish the optimal nonergodic sublinear convergence rate of the proximal point algorithm for maximal monotone inclusion problems. First, the optimal bound is formulated by the performance estimation framework, resulting in an infinite dimensional nonconvex optimization problem, which is then equivalently reformulated as a finite dimensional semidefinite programming (SDP) problem. By constructing a feasible solution to the dual SDP, we obtain an upper bound on the optimal nonergodic sublinear rate. Finally, an example in two dimensional space is constructed to provide a lower bound on the optimal nonergodic sublinear rate. Since the lower bound provided by the example matches exactly the upper bound obtained by the dual SDP, we have thus established the worst case nonergodic sublinear convergence rate which is optimal in terms of both the order as well as the constants involved. Our result sharpens the understanding of the fundamental proximal point algorithm.Comment: 15 pages, 2 figure

On the optimal linear convergence factor of the relaxed proximal point algorithm for monotone inclusion problems

Finding a zero of a maximal monotone operator is fundamental in convex optimization and monotone operator theory, and \emph{proximal point algorithm} (PPA) is a primary method for solving this problem. PPA converges not only globally under fairly mild conditions but also asymptotically at a fast linear rate provided that the underlying inverse operator is Lipschitz continuous at the origin. These nice convergence properties are preserved by a relaxed variant of PPA. Recently, a linear convergence bound was established in [M. Tao, and X. M. Yuan, J. Sci. Comput., 74 (2018), pp. 826-850] for the relaxed PPA, and it was shown that the bound is optimal when the relaxation factor $\gamma$ lies in $[1,2)$. However, for other choices of $\gamma$, the bound obtained by Tao and Yuan is suboptimal. In this paper, we establish tight linear convergence bounds for any choice of $\gamma\in(0,2)$ and make the whole picture about optimal linear convergence bounds clear. These results sharpen our understandings to the asymptotic behavior of the relaxed PPA.Comment: 9 pages and 1 figur

Applications of gauge duality in robust principal component analysis and semidefinite programming

Gauge duality theory was originated by Freund [Math. Programming, 38(1):47-67, 1987] and was recently further investigated by Friedlander, Mac{\^e}do and Pong [SIAM J. Optm., 24(4):1999-2022, 2014]. When solving some matrix optimization problems via gauge dual, one is usually able to avoid full matrix decompositions such as singular value and/or eigenvalue decompositions. In such an approach, a gauge dual problem is solved in the first stage, and then an optimal solution to the primal problem can be recovered from the dual optimal solution obtained in the first stage. Recently, this theory has been applied to a class of \emph{semidefinite programming} (SDP) problems with promising numerical results [Friedlander and Mac{\^e}do, SIAM J. Sci. Comp., to appear, 2016]. In this paper, we establish some theoretical results on applying the gauge duality theory to robust \emph{principal component analysis} (PCA) and general SDP. For each problem, we present its gauge dual problem, characterize the optimality conditions for the primal-dual gauge pair, and validate a way to recover a primal optimal solution from a dual one. These results are extensions of [Friedlander and Mac{\^e}do, SIAM J. Sci. Comp., to appear, 2016] from nuclear norm regularization to robust PCA and from a special class of SDP which requires the coefficient matrix in the linear objective to be positive definite to SDP problems without this restriction. Our results provide further understanding in the potential advantages and disadvantages of the gauge duality theory.Comment: Accepted for publication in SCIENCE CHINA Mathematic

Alternating Direction Algorithms for ℓ1\ell_1-Problems in Compressive Sensing

In this paper, we propose and study the use of alternating direction algorithms for several $\ell_1$-norm minimization problems arising from sparse solution recovery in compressive sensing, including the basis pursuit problem, the basis-pursuit denoising problems of both unconstrained and constrained forms, as well as others. We present and investigate two classes of algorithms derived from either the primal or the dual forms of the $\ell_1$-problems. The construction of the algorithms consists of two main steps: (1) to reformulate an $\ell_1$-problem into one having partially separable objective functions by adding new variables and constraints; and (2) to apply an exact or inexact alternating direction method to the resulting problem. The derived alternating direction algorithms can be regarded as first-order primal-dual algorithms because both primal and dual variables are updated at each and every iteration. Convergence properties of these algorithms are established or restated when they already exist. Extensive numerical results in comparison with several state-of-the-art algorithms are given to demonstrate that the proposed algorithms are efficient, stable and robust. Moreover, we present numerical results to emphasize two practically important but perhaps overlooked points. One point is that algorithm speed should always be evaluated relative to appropriate solution accuracy; another is that whenever erroneous measurements possibly exist, the $\ell_1$-norm fidelity should be the fidelity of choice in compressive sensing

Capturing Near Earth Objects

Recently, Near Earth Objects (NEOs) have been attracting great attention, and thousands of NEOs have been found to date. This paper examines the NEOs' orbital dynamics using the framework of an accurate solar system model and a Sun-Earth-NEO three-body system when the NEOs are close to Earth to search for NEOs with low-energy orbits. It is possible for such an NEO to be temporarily captured by Earth; its orbit would thereby be changed and it would become an Earth-orbiting object after a small increase in its velocity. From the point of view of the Sun-Earth-NEO restricted three-body system, it is possible for an NEO whose Jacobian constant is slightly lower than C1 and higher than C3 to be temporarily captured by Earth. When such an NEO approaches Earth, it is possible to change its orbit energy to close up the zero velocity surface of the three-body system at point L1 and make the NEO become a small satellite of the Earth. Some such NEOs were found; the best example only required a 410m/s increase in velocity.Comment: 22 pages, 6 figures, accepted for publication in Research in Astronomy and Astrophysics (Chinese Journal of Astronomy and Astrophysics

A general inertial proximal point method for mixed variational inequality problem

In this paper, we first propose a general inertial proximal point method for the mixed variational inequality (VI) problem. Based on our knowledge, without stronger assumptions, convergence rate result is not known in the literature for inertial type proximal point methods. Under certain conditions, we are able to establish the global convergence and a $o(1/k)$ convergence rate result (under certain measure) of the proposed general inertial proximal point method. We then show that the linearized alternating direction method of multipliers (ADMM) for separable convex optimization with linear constraints is an application of a general proximal point method, provided that the algorithmic parameters are properly chosen. As byproducts of this finding, we establish global convergence and $O(1/k)$ convergence rate results of the linearized ADMM in both ergodic and nonergodic sense. In particular, by applying the proposed inertial proximal point method for mixed VI to linearly constrained separable convex optimization, we obtain an inertial version of the linearized ADMM for which the global convergence is guaranteed. We also demonstrate the effect of the inertial extrapolation step via experimental results on the compressive principal component pursuit problem.Comment: 21 pages, two figures, 4 table

EOS: Automatic In-vivo Evolution of Kernel Policies for Better Performance

Today's monolithic kernels often implement a small, fixed set of policies such as disk I/O scheduling policies, while exposing many parameters to let users select a policy or adjust the specific setting of the policy. Ideally, the parameters exposed should be flexible enough for users to tune for good performance, but in practice, users lack domain knowledge of the parameters and are often stuck with bad, default parameter settings. We present EOS, a system that bridges the knowledge gap between kernel developers and users by automatically evolving the policies and parameters in vivo on users' real, production workloads. It provides a simple policy specification API for kernel developers to programmatically describe how the policies and parameters should be tuned, a policy cache to make in-vivo tuning easy and fast by memorizing good parameter settings for past workloads, and a hierarchical search engine to effectively search the parameter space. Evaluation of EOS on four main Linux subsystems shows that it is easy to use and effectively improves each subsystem's performance.Comment: 14 pages, technique repor

Optimal Bidding Algorithms Against Cheating in Multiple-Object Auctions

This paper studies some basic problems in a multiple-object auction model using methodologies from theoretical computer science. We are especially concerned with situations where an adversary bidder knows the bidding algorithms of all the other bidders. In the two-bidder case, we derive an optimal randomized bidding algorithm, by which the disadvantaged bidder can procure at least half of the auction objects despite the adversary's a priori knowledge of his algorithm. In the general $k$-bidder case, if the number of objects is a multiple of $k$, an optimal randomized bidding algorithm is found. If the $k-1$ disadvantaged bidders employ that same algorithm, each of them can obtain at least $1/k$ of the objects regardless of the bidding algorithm the adversary uses. These two algorithms are based on closed-form solutions to certain multivariate probability distributions. In situations where a closed-form solution cannot be obtained, we study a restricted class of bidding algorithms as an approximation to desired optimal algorithms

Inertial primal-dual algorithms for structured convex optimization

The primal-dual algorithm recently proposed by Chambolle & Pock (abbreviated as CPA) for structured convex optimization is very efficient and popular. It was shown by Chambolle & Pock in \cite{CP11} and also by Shefi & Teboulle in \cite{ST14} that CPA and variants are closely related to preconditioned versions of the popular alternating direction method of multipliers (abbreviated as ADM). In this paper, we further clarify this connection and show that CPAs generate exactly the same sequence of points with the so-called linearized ADM (abbreviated as LADM) applied to either the primal problem or its Lagrangian dual, depending on different updating orders of the primal and the dual variables in CPAs, as long as the initial points for the LADM are properly chosen. The dependence on initial points for LADM can be relaxed by focusing on cyclically equivalent forms of the algorithms. Furthermore, by utilizing the fact that CPAs are applications of a general weighted proximal point method to the mixed variational inequality formulation of the KKT system, where the weighting matrix is positive definite under a parameter condition, we are able to propose and analyze inertial variants of CPAs. Under certain conditions, global point-convergence, nonasymptotic $O(1/k)$ and asymptotic $o(1/k)$ convergence rate of the proposed inertial CPAs can be guaranteed, where $k$ denotes the iteration index. Finally, we demonstrate the profits gained by introducing the inertial extrapolation step via experimental results on compressive image reconstruction based on total variation minimization.Comment: 23 pages, 4 figures, 3 table

Photolysis of n-butyl nitrite and isoamyl nitrite at 355 nm: A time-resolved Fourier transform infrared emission spectroscopy and ab initio study

We report on the photodissociation dynamics study of n-butyl nitrite (n-C_4H_9ONO) and isoamyl nitrite ((CH_3)_2C_3H_5ONO) by means of time-resolved Fourier transform infrared (TR-FTIR) emission spectroscopy. The obtained TR-FTIR emission spectra of the nascent NO fragments produced in the 355-nm laser photolysis of the two alkyl nitrite species showed an almost identical rotational temperature and vibrational distributions of NO. In addition, a close resemblance between the two species was also found in the measured temporal profiles of the IR emission of NO and the recorded UV absorption spectra. The experimental results are consistent with our ab initio calculations using the time-dependent density functional theory at the B3LYP/6-311G(d,p) level, which indicate that the substitution of one of the two {gamma}-H atoms in n-C_4H_9ONO with a methyl group to form (CH_3)_2C_3H_5ONO has only a minor effect on the photodissociation dynamics of the two molecules.Comment: 25 pages, 7 figures, 3 tables; submitted to JC